A light-field camera, also called a plenoptic camera, is a camera that uses a microlens array to capture 4D light field information about a scene. Such light field information can be used to improve the solution of computer graphics and vision-related problems.
Contents |
In 1992, Adelson and Wang[1] proposed the design of a plenoptic camera that can be used to significantly reduce the correspondence problem in stereo matching. To achieve this, an array of microlenses is placed at the focal plane of the camera main lens. The image sensor is positioned slightly behind the microlenses. Using such images the displacement of image parts that are not in focus can be analyzed and depth information can be extracted. Potentially, this camera system can be used to refocus an image virtually on a computer after the picture has been taken, as explained by Ng et al.[2] The drawback of such a system is the low resolution that the final images have. As one microlens samples the light directions at one spatial point an increase in the number of image pixels can only be done by increasing the number of microlenses by the same amount. To overcome this limitation, Lumsdaine and Georgiev[3] describe a new design of a plenoptic camera, called focused plenoptic camera where the microlens array is positioned before or behind the focal plane of the main lens. This modification samples the light field in a different way that allows for a higher spatial resolution by having a lower angular resolution at the same time. With this design images can be refocused with a much higher spatial resolution. However, the low angular resolution can introduce some unwanted aliasing artifacts. A different design[4] using low cost printed film (mask) instead of microlens array was proposed by researchers at MERL in 2007. This design overcomes several limitations of microlens array in terms of chromatic aberrations, loss of boundary pixels, and allows higher spatial resolution photos to be captured. However, a mask based design reduces light compared to microlens arrays.
In 2004, a team at Stanford University Computer Graphics Laboratory used a 16 megapixel camera with a 90,000-microlens array (meaning that each microlens covers about 175 pixels, and the final resolution is 90 kilopixels) to demonstrate that pictures can be refocused after they are taken.[2]
Plenoptic cameras are good for following moving objects.[5] A recording from a security camera based upon this technology could be used to produce an accurate 3D model of a suspect.[6]
From 2010 Raytrix has sold several models of plenoptic camera for industrial and scientific applications, with resolutions starting from 1 megapixel[7][8].
Lytro was founded by Stanford University Computer Graphics Laboratory alumnus Ren Ng to commercialize the light field camera he developed as a graduate student there. Lytro has developed a consumer light-field "digital camera capable of images using a plenoptic technique[9]. Orders were being taken in late 2011 for cameras to be delivered early in 2012.
The Adobe light-field camera is a prototype 100-megapixel camera that takes a three-dimensional photo of the scene in focus using 19 uniquely configured lenses. Each lens will take a 5.2 megapixel photo of the entire scene around the camera and each image can be focused later in any way.[10]
The CAFADIS camera[11] is a plenoptic camera developed by the University of La Laguna (Spain). CAFADIS stands (in Spanish) for phase-distance camera, since it can be used for distance and optical wavefront estimation. From a single shot it can produce several images refocused at different distances, depth maps, all-in-focus images and stereo pairs. A similar optical design can also be used in adaptive optics in astrophysics, in order to correct the aberrations caused by atmospheric turbulence in telescope images. In order to perform these tasks different algorithms, running on GPU and FPGA, operate on the raw image captured by the camera.
Mitsubishi Electric Research Labs's (MERL) light field camera[12] is based on the principle of optical heterodyning and uses a printed film (mask) placed close to the sensor. Any hand-held camera can be converted into a light field camera using this technology by simply inserting a low-cost film on top of the sensor.[13] A mask-based design avoids the problem of loss of resolution, since a high-resolution photo can be generated for the focused parts of the scene.
Stanford University Computer Graphics Laboratory has developed a light-field microscope using a microlens array similar to the one used in the light-field camera developed by the lab. The prototype is built around a Nikon Eclipse transmitted light microscope/wide-field fluorescence microscope and standard CCD cameras. Light-field capturing ability is obtained by a module containing a microlens array and other optical components placed in the light path between the objective lens and camera, with the final multifocused image rendered using deconvolution.[14][15][16] A later version of the prototype added a light-field illumination system consisting of a video projector (allowing computational control of illumination) and a second microlens array in the illumination light path of the microscope. The addition of a light-field illumination system both allowed for additional types of illumination (such as oblique illumination and quasi-dark-field) and correction for optical aberrations.[15]